Assessment of pulmonary vascular anatomy: comparing augmented reality by holograms versus standard CT images/reconstructions using surgical findings as reference standard.

BACKGROUND: We compared computed tomography (CT) images and holograms (HG) to assess the number of arteries of the lung lobes undergoing lobectomy and assessed easiness in interpretation by radiologists and thoracic surgeons with both techniques. METHODS: Patients scheduled for lobectomy for lung cancer were prospectively included and underwent CT for staging. A patient-specific three-dimensional model was generated and visualized in an augmented reality setting. One radiologist and one thoracic surgeon evaluated CT images and holograms to count lobar arteries, having as reference standard the number of arteries recorded at surgery. The easiness of vessel identification was graded according to a Likert scale. Wilcoxon signed-rank test and κ statistics were used. RESULTS: Fifty-two patients were prospectively included. The two doctors detected the same number of arteries in 44/52 images (85%) and in 51/52 holograms (98%). The mean difference between the number of artery branches detected by surgery and CT images was 0.31 ± 0.98, whereas it was 0.09 ± 0.37 between surgery and HGs (p = 0.433). In particular, the mean difference in the number of arteries detected in the upper lobes was 0.67 ± 1.08 between surgery and CT images and 0.17 ± 0.46 between surgery and holograms (p = 0.029). Both radiologist and surgeon showed a higher agreement for holograms (κ = 0.99) than for CT (κ = 0.81) and found holograms easier to evaluate than CTs (p < 0.001). CONCLUSIONS: Augmented reality by holograms is an effective tool for preoperative vascular anatomy assessment of lungs, especially when evaluating the upper lobes, more prone to anatomical variations. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04227444 RELEVANCE STATEMENT: Preoperative evaluation of the lung lobe arteries through augmented reality may help the thoracic surgeons to carefully plan a lobectomy, thus contributing to optimize patients' outcomes. KEY POINTS: • Preoperative assessment of the lung arteries may help surgical planning. • Lung artery detection by augmented reality was more accurate than that by CT images, particularly for the upper lobes. • The assessment of the lung arterial vessels was easier by using holograms than CT images.

Effect of sound-induced vibrations of the pinna on head-related transfer functions: Experimental and numerical investigations.

Numerical simulations of head-related transfer functions (HRTFs) conventionally assume a rigid boundary condition for the pinna. The human pinna, however, is an elastic deformable body that can vibrate due to incident acoustic waves. This work investigates how sound-induced vibrations of the pinna can affect simulated HRTF magnitudes. The work will motivate the research question by measuring the sound-induced vibrational patterns of an artificial pinna with a high-speed holographic interferometric system. Then, finite element simulations are used to determine HRTFs for a tabletop model of the B&K 5128 head and torso simulator for a number of directions. Two scenarios are explored: one where the pinna is modeled as perfectly rigid, and another where the pinna is modeled as linear elastic with material properties close to that of auricular cartilage. The findings suggest that pinna vibrations have negligible effects on HRTF magnitudes up to 5 kHz. The same conclusion, albeit with less certainty, is drawn for higher frequencies. Finally, the importance of the elastic domain's material properties is emphasized and possible implications for validation studies on dummy heads 1as well as the limitations of the present work are discussed in detail.

Quantitative Phase Imaging as Sensitive Screening Method for Nanoparticle-Induced Cytotoxicity Assessment.

The assessment of nanoparticle cytotoxicity is challenging due to the lack of customized and standardized guidelines for nanoparticle testing. Nanoparticles, with their unique properties, can interfere with biochemical test methods, so multiple tests are required to fully assess their cellular effects. For a more reliable and comprehensive assessment, it is therefore imperative to include methods in nanoparticle testing routines that are not affected by particles and allow for the efficient integration of additional molecular techniques into the workflow. Digital holographic microscopy (DHM), an interferometric variant of quantitative phase imaging (QPI), has been demonstrated as a promising method for the label-free assessment of the cytotoxic potential of nanoparticles. Due to minimal interactions with the sample, DHM allows for further downstream analyses. In this study, we investigated the capabilities of DHM in a multimodal approach to assess cytotoxicity by directly comparing DHM-detected effects on the same cell population with two downstream biochemical assays. Therefore, the dry mass increase in RAW 264.7 macrophages and NIH-3T3 fibroblast populations measured by quantitative DHM phase contrast after incubation with poly(alkyl cyanoacrylate) nanoparticles for 24 h was compared to the cytotoxic control digitonin, and cell culture medium control. Viability was then determined using a metabolic activity assay (WST-8). Moreover, to determine cell death, supernatants were analyzed for the release of the enzyme lactate dehydrogenase (LDH assay). In a comparative analysis, in which the average half-maximal effective concentration (EC50) of the nanocarriers on the cells was determined, DHM was more sensitive to the effect of the nanoparticles on the used cell lines compared to the biochemical assays.

Real-time 3D tracking of swimming microbes using digital holographic microscopy and deep learning.

The three-dimensional swimming tracks of motile microorganisms can be used to identify their species, which holds promise for the rapid identification of bacterial pathogens. The tracks also provide detailed information on the cells' responses to external stimuli such as chemical gradients and physical objects. Digital holographic microscopy (DHM) is a well-established, but computationally intensive method for obtaining three-dimensional cell tracks from video microscopy data. We demonstrate that a common neural network (NN) accelerates the analysis of holographic data by an order of magnitude, enabling its use on single-board computers and in real time. We establish a heuristic relationship between the distance of a cell from the focal plane and the size of the bounding box assigned to it by the NN, allowing us to rapidly localise cells in three dimensions as they swim. This technique opens the possibility of providing real-time feedback in experiments, for example by monitoring and adapting the supply of nutrients to a microbial bioreactor in response to changes in the swimming phenotype of microbes, or for rapid identification of bacterial pathogens in drinking water or clinical samples.

[Simulation and Navigation in the Neurosurgical Field Using Three-Dimensional Hholograms by Mixed Reality Devices].

Recently, three-dimensional(3D)holograms from mixed-reality(MR)devices have become available in the medical field. 3D holographic images can provide immersive and intuitive information that has been reported to be very useful for preoperative simulations. Compared with conventional 3D images on a two-dimensional(2D)monitor, 3D holograms offer a higher level of realism, allowing observation of the images anytime and anywhere if the MR device is operational. Even during surgery, surgeons can check realistic 3D holograms in front of them, above the surgical field, without having to turn their heads toward a 2D monitor on the wall. 3D holograms can also be used for neuronavigation if the hologram is tracked to the patient's real head. This method can be defined as 3D augmented reality(AR)navigation, which shows a hologram of a target, such as a tumor or aneurysm, inside the head and brain. In the future, interventions using these techniques with 3D holograms from MR devices are expected to evolve and develop new types of treatments for endoscopic surgery or fluoroscopy-guided endovascular surgery.

Comparative oncology chemosensitivity assay for personalized medicine using low-coherence digital holography of dynamic light scattering from cancer biopsies.

Nearly half of cancer patients who receive standard-of-care treatments fail to respond to their first-line chemotherapy, demonstrating the pressing need for improved methods to select personalized cancer therapies. Low-coherence digital holography has the potential to fill this need by performing dynamic contrast OCT on living cancer biopsies treated ex vivo with anti-cancer therapeutics. Fluctuation spectroscopy of dynamic light scattering under conditions of holographic phase stability captures ultra-low Doppler frequency shifts down to 10 mHz caused by light scattering from intracellular motions. In the comparative preclinical/clinical trials presented here, a two-species (human and canine) and two-cancer (esophageal carcinoma and B-cell lymphoma) analysis of spectral phenotypes identifies a set of drug response characteristics that span species and cancer type. Spatial heterogeneity across a centimeter-scale patient biopsy sample is assessed by measuring multiple millimeter-scale sub-samples. Improved predictive performance is achieved for chemoresistance profiling by identifying red-shifted sub-samples that may indicate impaired metabolism and removing them from the prediction analysis. These results show potential for using biodynamic imaging for personalized selection of cancer therapy.

Live Cell Imaging by Single-Shot Common-Path Wide Field-of-View Reflective Digital Holographic Microscope.

Quantitative phase imaging by digital holographic microscopy (DHM) is a nondestructive and label-free technique that has been playing an indispensable role in the fields of science, technology, and biomedical imaging. The technique is competent in imaging and analyzing label-free living cells and investigating reflective surfaces. Herein, we introduce a new configuration of a wide field-of-view single-shot common-path off-axis reflective DHM for the quantitative phase imaging of biological cells that leverages several advantages, including being less-vibration sensitive to external perturbations due to its common-path configuration, also being compact in size, simple in optical design, highly stable, and cost-effective. A detailed description of the proposed DHM system, including its optical design, working principle, and capability for phase imaging, is presented. The applications of the proposed system are demonstrated through quantitative phase imaging results obtained from the reflective surface (USAF resolution test target) as well as transparent samples (living plant cells). The proposed system could find its applications in the investigation of several biological specimens and the optical metrology of micro-surfaces.

Label-free cell classification in holographic flow cytometry through an unbiased learning strategy.

Nowadays, label-free imaging flow cytometry at the single-cell level is considered the stepforward lab-on-a-chip technology to address challenges in clinical diagnostics, biology, life sciences and healthcare. In this framework, digital holography in microscopy promises to be a powerful imaging modality thanks to its multi-refocusing and label-free quantitative phase imaging capabilities, along with the encoding of the highest information content within the imaged samples. Moreover, the recent achievements of new data analysis tools for cell classification based on deep/machine learning, combined with holographic imaging, are urging these systems toward the effective implementation of point of care devices. However, the generalization capabilities of learning-based models may be limited from biases caused by data obtained from other holographic imaging settings and/or different processing approaches. In this paper, we propose a combination of a Mask R-CNN to detect the cells, a convolutional auto-encoder, used to the image feature extraction and operating on unlabelled data, thus overcoming the bias due to data coming from different experimental settings, and a feedforward neural network for single cell classification, that operates on the above extracted features. We demonstrate the proposed approach in the challenging classification task related to the identification of drug-resistant endometrial cancer cells.

Comparison of holotomographic microscopy and coherence-controlled holographic microscopy.

Quantitative phase imaging (QPI) is a powerful tool for label-free visualisation of living cells. Here, we compare two QPI microscopes - the Telight Q-Phase microscope and the Nanolive 3D Cell Explorer-fluo microscope. Both systems provide unbiased information about cell morphology, such as individual cell dry mass, perimeter and area. The Q-Phase microscope uses artefact-free, coherence-controlled holographic imaging technology to visualise cells in real time with minimal phototoxicity. The 3D Cell Explorer-fluo employs laser-based holotomography to reconstruct 3D images of living cells, visualising their internal structures and dynamics. Here, we analysed the strengths and limitations of both microscopes when examining two morphologically distinct cell lines - the cuboidal epithelial MDCK cells which form multicellular clusters and solitary growing Rat2 fibroblasts. We focus mainly on the ability of the devices to generate images suitable for single-cell segmentation by the built-in software, and we discuss the segmentation results and quantitative data generated from the segmented images. We show that both microscopes offer slightly different advantages, and the choice between them depends on the specific requirements and goals of the user.

A dataset of digital holograms of normal and thalassemic cells.

Digital holographic microscopy (DHM) is an intriguing medical diagnostic tool due to its label-free and quantitative nature, providing high-contrast images of phase samples. By capturing both intensity and phase information, DHM enables the numerical reconstruction of quantitative phase images. However, the lateral resolution is limited by the diffraction limit, which prompted the recent suggestion of microsphere-assisted DHM to enhance the DHM resolution straightforwardly. The use of such a technique as a medical diagnostic tool requires testing and validation of the proposed assays to prove their feasibility and viability. This paper publishes 760 and 609 microsphere-assisted DHM images of normal and thalassemic red blood cells obtained from a normal and thalassemic male individual, respectively.

Moving grating-based laser heterodyne digital holographic microscopy system for measuring dynamic phase of living cell attachment.

We propose a laser heterodyne digital holography microscopy system based on a moving grating, which uses the Doppler principle between a moving grating and beam to achieve a low-frequency bias between the diffracted beams, abandoning traditional heterodyne digital holography that requires multiple acousto-optic modulators. The dynamic phase distribution obtained using the laser heterodyne digital holography phase-reconstruction algorithm was more realistic and analyzable than the results of the angular spectrum algorithm. The structure and algorithm were used to capture the shape characteristics of mouse fibroblasts after ~2 h of incubation (37°C, 5% CO2), and the dynamic phase distribution of the cells was monitored in real-time during the attachment process. The system proposed in this study, with its high spatial resolution and high-precision phase measurement capability, is suitable for both static and live cells.

Fast 2-photon stimulation using holographic patterns.

Two decades after its introduction, optogenetics - a biological technique to control the activity of neurons or other cell types with light - remains a cutting edge and promising tool to study biological processes. Its increasing usage in research varies widely from causally exploring biological mechanisms and neural computations, to neurostimulation and sensory restauration. To stimulate neurons in the brain, a variety of approaches have been developed to generate precise spatiotemporal light patterns. Yet certain constrains still exists in the current optical techniques to activate a neuronal population with both cellular resolution and millisecond precision. Here, we describe an experimental setup allowing to stimulate a few tens of neurons in a plane at sub-millisecond rates using 2-photon activation. A liquid crystal on silicon spatial light modulator (LCoS-SLM) was used to generate spatial patterns in 2 dimensions. The image of the patterns was formed on the plane of a digital micromirror device (DMD) that was used as a fast temporal modulator of each region of interest. Using fluorescent microscopy and patch-clamp recording of neurons in culture expressing the light-gated ion channels, we characterized the temporal and spatial resolution of the microscope. We described the advantages of combining the LCoS-SLM with the DMD to maximize the temporal precision, modulate the illumination amplitude, and reduce background activation. Finally, we showed that this approach can be extended to patterns in 3 dimensions. We concluded that the methodology is well suited to address important questions about the role of temporal information in neuronal coding.

Aberration-free holographic microscope for simultaneous imaging and stimulation of neuronal populations.

A technical challenge in neuroscience is to record and specifically manipulate the activity of neurons in living animals. This can be achieved in some preparations with two-photon calcium imaging and photostimulation. These methods can be extended to three dimensions by holographic light sculpting with spatial light modulators (SLMs). At the same time, performing simultaneous holographic imaging and photostimulation is still cumbersome, requiring two light paths with separate SLMs. Here we present an integrated optical design using a single SLM for simultaneous imaging and photostimulation. Furthermore, we applied axially dependent adaptive optics to make the system aberration-free, and developed software for calibrations and closed-loop neuroscience experiments. Finally, we demonstrate the performance of the system with simultaneous calcium imaging and optogenetics in mouse primary auditory cortex in vivo. Our integrated holographic system could facilitate the systematic investigation of neural circuit function in awake behaving animals.

On-chip label-free cell classification based directly on off-axis holograms and spatial-frequency-invariant deep learning.

We present a rapid label-free imaging flow cytometry and cell classification approach based directly on raw digital holograms. Off-axis holography enables real-time acquisition of cells during rapid flow. However, classification of the cells typically requires reconstruction of their quantitative phase profiles, which is time-consuming. Here, we present a new approach for label-free classification of individual cells based directly on the raw off-axis holographic images, each of which contains the complete complex wavefront (amplitude and quantitative phase profiles) of the cell. To obtain this, we built a convolutional neural network, which is invariant to the spatial frequencies and directions of the interference fringes of the off-axis holograms. We demonstrate the effectiveness of this approach using four types of cancer cells. This approach has the potential to significantly improve both speed and robustness of imaging flow cytometry, enabling real-time label-free classification of individual cells.

Randomness assisted in-line holography with deep learning.

We propose and demonstrate a holographic imaging scheme exploiting random illuminations for recording hologram and then applying numerical reconstruction and twin image removal. We use an in-line holographic geometry to record the hologram in terms of the second-order correlation and apply the numerical approach to reconstruct the recorded hologram. This strategy helps to reconstruct high-quality quantitative images in comparison to the conventional holography where the hologram is recorded in the intensity rather than the second-order intensity correlation. The twin image issue of the in-line holographic scheme is resolved by an unsupervised deep learning based method using an auto-encoder scheme. Proposed learning technique leverages the main characteristic of autoencoders to perform blind single-shot hologram reconstruction, and this does not require a dataset of samples with available ground truth for training and can reconstruct the hologram solely from the captured sample. Experimental results are presented for two objects, and a comparison of the reconstruction quality is given between the conventional inline holography and the one obtained with the proposed technique.

Functionally integrated waveguide illuminator for common-path digital holographic microscopy through random media.

We designed and fabricated a functionally integrated optical waveguide illuminator specially for common-path digital holographic microscopy through random media. The waveguide illuminator creates two point sources with desired phase shifts, which are located close to one another so that the common-path condition of the object and reference illumination is satisfied. Thereby, the proposed device permits phase-shift digital holographic microscopy free from bulky optical elements such as a beam splitter, an objective lens, and a piezoelectric transducer for phase shifting. Using the proposed device, microscopic 3D imaging through a highly heterogeneous double-composite random medium was experimentally demonstrated by means of common-path phase-shift digital holography.

Quantitative phase imaging of living red blood cells combining digital holographic microscopy and deep learning.

Digital holographic microscopy as a non-contacting, non-invasive, and highly accurate measurement technology, is becoming a valuable method for quantitatively investigating cells and tissues. Reconstruction of phases from a digital hologram is a key step in quantitative phase imaging for biological and biomedical research. This study proposes a two-stage deep convolutional neural network named VY-Net, to realize the effective and robust phase reconstruction of living red blood cells. The VY-Net can obtain the phase information of an object directly from a single-shot off-axis digital hologram. We also propose two new indices to evaluate the reconstructed phases. In experiments, the mean of the structural similarity index of reconstructed phases can reach 0.9309, and the mean of the accuracy of reconstructions of reconstructed phases is as high as 91.54%. An unseen phase map of a living human white blood cell is successfully reconstructed by the trained VY-Net, demonstrating its strong generality.

Polarimetric calibrated robust dual-SLM complex-amplitude computer-generated holography.

Liquid crystal on silicon (LCoS) is a widely used spatial light modulator (SLM) in computer-generated holography (CGH). However, the phase-modulating profile of LCoS is often not ideally uniform in application, bringing about undesired intensity fringes. In this study, we overcome this problem by proposing a highly robust dual-SLM complex-amplitude CGH technique, which incorporates a polarimetric mode and a diffractive mode. The polarimetric mode linearizes the general phase modulations of the two SLMs separately, while the diffractive mode uses camera-in-the-loop optimization to achieve improved holographic display. Experimental results show the effectiveness of our proposal in improving reconstructing accuracy by 21.12% in peak signal-to-noise ratio (PSNR) and 50.74% in structure similarity index measure (SSIM), using LCoS SLMs with originally non-uniform phase-modulating profiles.

Adaptive constraints by morphological operations for single-shot digital holography.

Digital holography provides access to quantitative measurement of the entire complex field, which is indispensable for the investigation of wave-matter interactions. The emerging iterative phase retrieval approach enables to solve the inverse imaging problem only from the given intensity measurements and physical constraints. However, enforcing imprecise constraints limits the reconstruction accuracy and convergence speed. Here, we propose an advanced iterative phase retrieval framework for single-shot in-line digital holography that incorporates adaptive constraints, which achieves optimized convergence behavior, high-fidelity and twin-image-free reconstruction. In conjunction with morphological operations which can extract the object structure while eliminating the irrelevant part such as artifacts and noise, adaptive constraints allow the support region to be accurately estimated and automatically updated at each iteration. Numerical reconstruction of complex-valued objects and the capability of noise immunity are investigated. The improved reconstruction performance of this approach is experimentally validated. Such flexible and versatile framework has promising applications in biomedicine, X-ray coherent diffractive imaging and wavefront sensing.

Ultrafast light targeting for high-throughput precise control of neuronal networks.

Two-photon, single-cell resolution optogenetics based on holographic light-targeting approaches enables the generation of precise spatiotemporal neuronal activity patterns and thus a broad range of experimental applications, such as high throughput connectivity mapping and probing neural codes for perception. Yet, current holographic approaches limit the resolution for tuning the relative spiking time of distinct cells to a few milliseconds, and the achievable number of targets to 100-200, depending on the working depth. To overcome these limitations and expand the capabilities of single-cell optogenetics, we introduce an ultra-fast sequential light targeting (FLiT) optical configuration based on the rapid switching of a temporally focused beam between holograms at kHz rates. We used FLiT to demonstrate two illumination protocols, termed hybrid- and cyclic-illumination, and achieve sub-millisecond control of sequential neuronal activation and high throughput multicell illumination in vitro (mouse organotypic and acute brain slices) and in vivo (zebrafish larvae and mice), while minimizing light-induced thermal rise. These approaches will be important for experiments that require rapid and precise cell stimulation with defined spatio-temporal activity patterns and optical control of large neuronal ensembles.